Approximations of Predictive Entropy Correlate with Reading Times

The lexical frequency of an upcoming word affects reading times even when the upcoming word is masked from readers (Angele et al., 2015). One explanation for this observation is that readers may slow down if there is high uncertainty about upcoming material. In line with this hypothesis, this study finds a positive correlation between predictive entropy and self-paced reading times. This study also demonstrates that such predictive entropy can be effectively approximated by the surprisal of upcoming observations and that this future surprisal estimate is more predictive of reading times when the grammar is more granular, which would be prohibitively expensive for predictive entropy. These results suggest readers engage in fine-grained predictive estimations of certainty about upcoming lexical and syntactic material, that such predictions influence reading times, and that estimating that uncertainty can be done less expensively and more robustly with information-theoretic surprisal.